Conversation
|
📦 Preview · View → · 🟢 Live Verified for |
8c59ee1 to
3886444
Compare
…#190) * Add MySQL integration documentation and from_mysql operator reference Adds comprehensive documentation for the from_mysql operator, including: - New integration page at /integrations/mysql - Detailed operator reference at /reference/operators/from_mysql with parameter descriptions, type mappings, and usage examples - Updates to integration and operator index pages to include MySQL The from_mysql operator enables reading data from MySQL databases using table selection, raw SQL queries, or metadata inspection. Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com> * Finalize docs --------- Co-authored-by: Claude Opus 4.5 <noreply@anthropic.com>
* Add from_tcp operator documentation Document the new from_tcp operator with TLS support, including the tls record parameter, peer variable, and usage examples. Update the TCP integration page and cross-reference from_tcp in load_tcp and save_tcp See Also sections. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * Fix from_tcp docs: use $peer syntax and correct types Use $peer instead of peer for let-bound variable references in sub-pipelines. Correct the ip field type from string to ip and the port field type from int to int64. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
Document the `live` and `tracking_column` parameters that enable continuous polling for new rows using watermark tracking on an integer column. Add examples for both auto-detected and explicit tracking columns. Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
Add note explaining that internal metadata queries use MySQL prepared statements to prevent SQL injection through user-provided table and column names. Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
588262f to
17b8b2b
Compare
* Add read_tql operator documentation Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * Use from_file with read_tql in examples Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * Replace round-trip example with schema and selector examples Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * Revert "Replace round-trip example with schema and selector examples" This reverts commit b2f4078. * Remove round-trip example Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
📝 WalkthroughWalkthroughAdds MySQL integration docs and diagram; introduces new operators (from_stdin, from_mysql, read_tql, accept_tcp, to_tcp, serve_tcp, from_tcp); updates many TCP-related docs to new accept/to/serve operator forms; adjusts several operator docs (batch, measure, parallel) and sidebar to include MySQL. Changes
Sequence Diagram(s)mermaid Estimated code review effort🎯 3 (Moderate) | ⏱️ ~22 minutes Poem
🚥 Pre-merge checks | ✅ 2 | ❌ 1❌ Failed checks (1 inconclusive)
✅ Passed checks (2 passed)
✏️ Tip: You can configure your own custom pre-merge checks in the settings. ✨ Finishing Touches🧪 Generate unit tests (beta)
📝 Coding Plan
Comment |
There was a problem hiding this comment.
Actionable comments posted: 4
Caution
Some comments are outside the diff and can’t be posted inline due to platform limitations.
⚠️ Outside diff range comments (1)
src/content/docs/reference/operators/measure.md (1)
39-90: 🧹 Nitpick | 🔵 TrivialConsider adding an example demonstrating the
cumulativeparameter.The examples effectively show basic usage of
measure, but none demonstrate thecumulativeparameter—the only configurable option for this operator. Adding an example withcumulative=truewould help users understand the difference between per-batch statistics and running totals.For instance, you could add an example showing how
measure cumulative=trueproduces running totals for events or bytes, making it clearer when to use this option versus post-processing withsummarize.🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed. In `@src/content/docs/reference/operators/measure.md` around lines 39 - 90, Add a new example demonstrating the cumulative parameter for the measure operator: show a short TQL snippet that loads a file (e.g., load_file "eve.json" / read_suricata) and runs measure with cumulative=true (measure cumulative=true) and include the expected output illustrating running totals (increasing events or bytes across batches) so readers can compare it to the existing non-cumulative examples; reference the measure operator and the cumulative=true flag and mention summarizing with summarize (e.g., summarize schema, events=sum(events)) as a contrast.
🤖 Prompt for all review comments with AI agents
Verify each finding against the current code and only fix it if needed.
Inline comments:
In `@src/content/docs/reference/operators.mdx`:
- Around line 362-373: Add the new operator entry to the operators index
frontmatter and the rendered card grid by inserting an operators list item with
name: 'from_tcp', description: 'Reads events from a TCP socket.', example:
'from_tcp "0.0.0.0:8090" { read_json }' and path: 'reference/operators/from_tcp'
into the frontmatter operators: array, and add a corresponding ReferenceCard
block for from_tcp in the grid (match the existing card structure used for
from_stdin/from_kafka/from_mysql) so the new reference/operators/from_tcp page
appears in the overview and card grid.
In `@src/content/docs/reference/operators/from_mysql.mdx`:
- Around line 24-29: The doc incorrectly states that show is mutually exclusive
with table; clarify that the three modes are: Table mode (use table to read
rows), SQL mode (use sql to run custom queries), and Show mode (use show to list
objects) but note that show is mutually exclusive with sql while some show
values (e.g., show="columns") require a table parameter; update the top summary
and the other occurrences referencing the modes (mentions of table/sql/show and
the example using show="columns") to explicitly state "show is exclusive with
sql, but show='columns' accepts/needs table='<name>'" and adjust the wording at
the other sections (lines referenced) accordingly so examples and constraints
are consistent with the parameter behavior.
- Around line 30-32: Update the sentence about MySQL prepared statements to
clarify they protect bound values, not identifiers: change the line mentioning
"prevent SQL injection through user-provided table and column names" to say that
internal metadata queries (e.g., resolving tracking columns for live mode)
validate or allowlist user-provided table and column names before executing
prepared statements, and that prepared statements are used to safely bind
values; for example: "Internal metadata queries (such as resolving tracking
columns for live mode) validate user-provided table and column names against a
safe list and then use MySQL prepared statements to safely bind values and
prevent SQL injection."
In `@src/content/docs/reference/operators/from_stdin.mdx`:
- Around line 30-40: Update the shell example so it actually runs the TQL
pipeline shown below: pipe the JSON into the tenzir CLI and invoke the TQL query
that uses from_stdin { read_json } (i.e. pass the TQL snippet to tenzir so the
input is parsed by read_json). Ensure the echo ... | tenzir invocation includes
the TQL query for from_stdin/read_json so the command is copy-paste runnable.
---
Outside diff comments:
In `@src/content/docs/reference/operators/measure.md`:
- Around line 39-90: Add a new example demonstrating the cumulative parameter
for the measure operator: show a short TQL snippet that loads a file (e.g.,
load_file "eve.json" / read_suricata) and runs measure with cumulative=true
(measure cumulative=true) and include the expected output illustrating running
totals (increasing events or bytes across batches) so readers can compare it to
the existing non-cumulative examples; reference the measure operator and the
cumulative=true flag and mention summarizing with summarize (e.g., summarize
schema, events=sum(events)) as a contrast.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Organization UI
Review profile: ASSERTIVE
Plan: Pro
Run ID: d95507d1-9d7b-4695-89c3-ccfd6f408c66
📒 Files selected for processing (16)
src/content/docs/integrations/index.mdxsrc/content/docs/integrations/mysql.excalidrawsrc/content/docs/integrations/mysql.mdxsrc/content/docs/integrations/tcp.mdxsrc/content/docs/reference/operators.mdxsrc/content/docs/reference/operators/batch.mdsrc/content/docs/reference/operators/from_mysql.mdxsrc/content/docs/reference/operators/from_stdin.mdxsrc/content/docs/reference/operators/from_tcp.mdxsrc/content/docs/reference/operators/load_tcp.mdxsrc/content/docs/reference/operators/measure.mdsrc/content/docs/reference/operators/parallel.mdxsrc/content/docs/reference/operators/read_tql.mdxsrc/content/docs/reference/operators/save_tcp.mdxsrc/content/docs/reference/operators/write_tql.mdxsrc/sidebar.ts
Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
There was a problem hiding this comment.
Actionable comments posted: 3
🤖 Prompt for all review comments with AI agents
Verify each finding against the current code and only fix it if needed.
Inline comments:
In `@src/content/docs/integrations/tcp.mdx`:
- Around line 39-45: In the "SSL/TLS" section remove the duplicated fragment
"like `certfile` and `keyfile`." so the sentence reads only once; locate the
repeated text under the SSL/TLS heading and delete the extra occurrence, then
verify spacing and punctuation around the remaining sentence to ensure the
paragraph reads cleanly.
In `@src/content/docs/reference/operators/save_tcp.mdx`:
- Around line 73-75: Update the "See Also" block to remove the deprecated
reference to load_tcp and replace it with the new migration targets—use to_tcp
and serve_tcp alongside from_tcp and keep the Integration tcp marker; locate the
See Also entries (symbols: from_tcp, load_tcp, Integration tcp) in save_tcp.mdx
and change load_tcp to to_tcp and/or serve_tcp to match the deprecation guidance
and the migration pattern (also consider updating the example referenced by the
example at Line 58 in a separate follow-up for consistency).
In `@src/content/docs/reference/operators/to_tcp.mdx`:
- Around line 29-32: Add a consistent TLS section header before the TLSOptions
partial: insert a "### tls = record (optional)" header immediately above where
TLSOptions is rendered (the <TLSOptions/> partial import/render in to_tcp.mdx)
so it matches the style used in accept_tcp.mdx and from_tcp.mdx; ensure the
header text exactly matches the other files for consistency.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Organization UI
Review profile: ASSERTIVE
Plan: Pro
Run ID: ebfdf061-7af7-4bff-a708-3be8c6917466
📒 Files selected for processing (21)
src/content/docs/explanations/configuration.mdxsrc/content/docs/guides/collecting/get-data-from-the-network.mdxsrc/content/docs/guides/node-setup/configure-tls.mdxsrc/content/docs/guides/routing/load-balance-pipelines.mdxsrc/content/docs/guides/routing/send-to-destinations.mdxsrc/content/docs/integrations/microsoft/windows-event-logs.mdxsrc/content/docs/integrations/syslog.mdxsrc/content/docs/integrations/tcp.mdxsrc/content/docs/reference/functions/secret.mdxsrc/content/docs/reference/operators/accept_tcp.mdxsrc/content/docs/reference/operators/from.mdxsrc/content/docs/reference/operators/from_tcp.mdxsrc/content/docs/reference/operators/load_balance.mdxsrc/content/docs/reference/operators/load_tcp.mdxsrc/content/docs/reference/operators/read_delimited_regex.mdxsrc/content/docs/reference/operators/read_gelf.mdxsrc/content/docs/reference/operators/read_lines.mdxsrc/content/docs/reference/operators/read_syslog.mdxsrc/content/docs/reference/operators/save_tcp.mdxsrc/content/docs/reference/operators/serve_tcp.mdxsrc/content/docs/reference/operators/to_tcp.mdx
Remove redundant TLS section headings and duplicate partial includes from the TCP and HTTP operator reference pages. This keeps the rendered operator docs from showing the same TLS options record twice.
Add from_tcp to the operator index and card grid, clarify from_mysql query mode and prepared statement wording, and remove duplicate TLS text from the TCP integration page.
Co-authored-by: Aljaž M. Eržen <aljaz@erzen.si>
d904d21 to
50e2dd9
Compare
Summary
Feature branch for the new executor documentation.
Individual sub-PRs will target this branch so they can be reviewed and merged
incrementally. Once all pieces are in place, this branch merges into
main.(Continuation of #192, moved to a topic branch after reverting the premature merge to
main.)Test plan
mainpasses build